NEW OPTIMIZATION METHODS AND APPLICATIONS IN KERNEL-BASED MACHINE LEARNING By ONUR ŞEREF A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy NEW OPTIMIZATION METHODS AND APPLICATIONS IN KERNEL-BASED MACHINE LEARNING By Onur Şeref December 2006 Chair: Panos M. Pardalos Major Department: Industrial and Systems Engineering In this study, new optimization methods are introduced on kernel-based machine learning. These novel methods solve real life classification problems, especially those arising in the biomedical area. The first main contribution of this study is the selective support vector machine (SelSVM) classifiers. SelSVM classifiers are motivated by the noisy temporal variations in the recordings of repeated cognitive processes, which affect the performance of standard support vector machine (SVM) classifiers. In the SelSVM classification problem there are sets of possible pattern vectors instead of individual pattern vectors. SelSVM classifiers select those pattern vectors from each set that would maximize the margin between the two classes of selected pattern vectors. SelSVM is compared with other standard alignment methods on a neural data set that is used for analyzing the integration of visual and motor cortexes in the primate brain. Selective kernel-based methods are then further extended to selective support vector regression (SelSVR). The second main contribution of this study is a fast classifier based on the standard generalized eigenvalue classifiers (GEC). The regularized GEC (ReGEC) uses a new regularization technique which reduces the solution of two eigenvalue problems in the original GEC to a single eigenvalue problem. A parallel implementation of ReGEC is developed to study large scale genomic problems. Finally, an incremental version I-ReGEC is developed to train large amounts of data efficiently. I-ReGEC incrementally builds a